In computer graphics, rendering refers to the conversion of data into a desired format for display on a display device. For example, it may be desirable to obtain an image of an object, define a viewpoint, and render a new image of the object from the viewpoint. Producing images of an object from different viewpoints may be useful in medical applications for disease diagnosis, disease staging, and surgical planning. In non-medical applications, such as engineering applications, it may be desirable to render images of objects from different viewpoints for analysis and design purposes.
One type of rendering used to process three-dimensional image data is referred to as volume rendering. Volume rendering involves defining a viewpoint and casting rays from the viewpoint to surfaces or regions of interest in the scene. Voxel color and intensity are accumulated or recorded at each point along each ray. The accumulated voxel color and intensity values are used to render the final image from the selected viewpoint. However, as will be described in more detail below, one problem with conventional volume rendering is that occluded surfaces or regions cannot be rendered in the final image, except from constrained viewpoints from which these surfaces or regions are viewable.
In medical applications, 3D display of medical data sets is an increasingly useful tool. Viewing 3D reconstructions of objects from magnetic resonance imaging (MRI) and computerized tomography (CT) data is more natural and intuitive than mentally reconstructing these objects from orthogonal slices of the data, especially with the increase in the size of datasets due to ever-increasing scanner resolutions. When displaying such data sets using volume rendering, appropriate selection of the transfer function is critical for determining which features of the data will be displayed. For applications such as virtual arthroscopy (VA), however, even a careful selection of an appropriate transfer function is not sufficient to display entire objects of interest within a joint from a single viewpoint. In VA, due to the close proximity of surfaces in a joint, radiologists have found existing methods lacking for obtaining desired views of features of interest. It may be desirable to enable radiologists to obtain views exterior to the space between the bone and cartilage surfaces (“joint space”) for viewing entire areas of interest. Such views are impossible using standard volume rendering because the surfaces occlude each other from viewpoints outside the joint space. Radiologists have found a similar problem when using standard volume rendering for virtual urography (VU) studies in which the physician wishes to obtain views of the interior of the kidney collecting system (a sack called the renal pelvis, the ureter, and the bladder). In this case, a view from outside the kidney collecting system showing the interior is impossible to obtain using standard volume rendering, as the outer surface occludes the inner volume.
Volume Rendering
Interactive volume rendering has had a long history since its first description.22 Kruger and Westermann describe hardware-accelerated volume renderings21 and Hadwiger et al. show how to perform rapid multi-technique rendering of presegmented volumetric data sets on commodity graphics hardware.11 Occlusion problems encountered when viewing entire volumes require classification and selection of which portions of the data to display.
The most general method of classification is specifying a transfer function to determine the color and opacity of voxels in the volume. The transfer function can be set to display isosurfaces in volume data.3, 23, 27 Recent research on volume rendering has focused on the design of complex transfer functions. Interactive17, 19 and automatic20 techniques have been developed for the construction of multidimensional transfer functions that incorporate first and second derivates to define surfaces.13, 18 Such work has led to an algorithm to automatically estimate opacity transfer functions to display tissue interfaces in noisy 3D ultrasound data.14 A parallel-coordinates interface that enables the user to efficiently search the large-dimensional parameter space, keeping track of the most effective settings and indicating nearby likely candidates is described by Tory et. al.33 However, none of these methods determine which portions of the volume to cull independently from the transfer function definition, retaining the full range of transfer function control.
When viewing volume data, unimportant portions of the volume often occlude areas of interest. A technique for view-dependent transparency, which aims to automatically produce translucent surfaces similar to technical illustrations, is described by Diepstraten et al.5 In a later work, techniques for automatically producing breakaway and cutaway illustrations of nested surfaces are described6. These illustrations remove portions of geometry that occlude the interior surfaces as opposed to rendering those portions as a translucent material. Most relevant to the VA problem is the work of Viola et al.,34 which produces importance-driven volume rendering that highlights features of interest and automatically subdues the display of occluding volumes using one of several possible techniques. These techniques are effective for pre-segmented volumes in which there exists a hierarchy of importance among objects. The problem of accelerating volume segmentation while rendering using graphics hardware has also been addressed.30 
Depth Peeling
Depth peeling is a frame-buffer-based technique that can achieve order-independent transparency when rendering polygons.4, 8, 24 Depth peeling techniques have been extended to texture-based volume rendering of isosurfaces, originating the term volumetric depth peeling.38 It may be desirable to extend these techniques to the more general case of ray-based volume rendering, retaining full transfer function control (not just isosurfaces), therefore enabling the full range of volumetric effects to be applied (including effects that simulate directionally-illuminated surface rendering).
Medical Applications
Volume rendering has been demonstrated to improve effectiveness in clinical settings compared to 2D and 3D views on film.12, 39 A volume-rendering-based interactive virtual colonoscopy navigation system is undergoing clinical validation.35 A combined surface and volume rendering approach for virtual colonoscopy that also indicates which region of the surface have been surveyed and which have not yet been examined has been described.16 Volume visualization systems have also been used for the study of aortic aneurysms.32 The feasibility of 3D rendering of joint surfaces and pathology using MRI and CT data sets has been demonstrated by several authors.1, 29, 36, 37 Such techniques provide real-time interaction and evaluation of joints for VA. However, this evaluation of joint surfaces is constrained to views within the joint space, as is the case in conventional arthroscopy. Fielding et al. describe a system for color coding hand-segmented bladder walls by thickness for tumor detection,9 a task similar to that of VU.
Viewing a joint surface from a viewpoint within the joint space is analogous to viewing a wall behind a furnace from the area between the wall and the furnace. From such a constrained viewpoint, many of the features of the wall are not easily seen. In order to obtain a better image of the wall, it may be desirable to render the image from a viewpoint within the furnace so that the portion of the furnace between the viewpoint and the wall is shown in a transparent or partially transparent manner. However, conventional imaging techniques have been limited to viewpoints from which the surface being rendered is visible.
Accordingly, there exists a need for improved methods, systems, and computer program products for processing three-dimensional image data to render an image from a viewpoint within or beyond an occluding region of the image data.